Add `settings` to Model base class #2136

svilupp · 2025-07-05T16:29:23Z

Fixes #2119

The implementation follows the suggestion by @DouweM

This PR:

implements a settings field in the Model class
adapts the settings resolution to go in this order: model settings < agent model settings < run-specific model settings
adds 2 updates to the docs to mention how to use this feature with FallbackModel and the new settings hierarchy
tests for the settings handling - it's in a separate file as it's a separate behavior, or should I merge it with other tests?

# Issues Encountered

Found various "special" model classes inheriting from the core Model superclass (instrumentedmodel, testmodel, wrappermodel, fallbackmodel)

I dediced on the following behaviour -- any thoughts?

Wrapper/Fallback Models: Should use the underlying model's settings, not have their own
Instrumented Models: Keep existing behaviour/fields unchanged (it literary has a field called settings, so I had to change the field definition to be self.settings: ModelSettings | InstrumentationSettings | None = settings
TestModel: Added settings field for API consistency

Next Steps

Agree the correct behavior/design for the complicated model classes
Perhaps I could refactor some wrapper code (tried to resolve the right settings in agent.run()) to be a separate utility with its own tests?

hyperlint-ai · 2025-07-05T16:29:39Z

PR Change Summary

Enhanced the Model base class by introducing a settings field and updating the documentation to reflect the new settings hierarchy.

Implemented a settings field in the Model class.
Established a settings resolution order: model settings, agent model settings, and run-specific model settings.
Updated documentation to explain the new settings feature and hierarchy.
Added tests for settings handling.

Modified Files

docs/agents.md
docs/models/index.md

How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

DouweM

@svilupp Thanks for working on this!

pydantic_ai_slim/pydantic_ai/models/__init__.py

DouweM · 2025-07-08T14:22:31Z

pydantic_ai_slim/pydantic_ai/agent.py

+        # Merge model settings from any Model
+        if isinstance(model_used, InstrumentedModel):
+            # For InstrumentedModel, get settings from the wrapped model
+            wrapped_model_settings = getattr(model_used.wrapped, 'settings', None)


Can we define settings on WrapperModel (which InstrumentedModel inherits from) to return self.wrapped.settings, so we don't need to do this special-casing?

That may require us to rename the field on Model to _settings, so that we can define settings as a property that can be overridden on WrapperModel.

DouweM · 2025-07-08T14:22:39Z

pydantic_ai_slim/pydantic_ai/agent.py

+                model_settings = merge_model_settings(wrapped_model_settings, model_settings)
+        else:
+            # For regular models, use their settings directly
+            current_settings = getattr(model_used, 'settings', None)


Can this be model_used.settings?

DouweM · 2025-07-08T14:25:49Z

pydantic_ai_slim/pydantic_ai/models/instrumented.py


    def __init__(
        self,
        wrapped: Model | KnownModelName,
        options: InstrumentationSettings | None = None,
    ) -> None:
        super().__init__(wrapped)
-        self.settings = options or InstrumentationSettings()
+        # Store instrumentation settings separately from model settings


I don't think we need this comment or the next one, with the different field names it's clear enough what's happening

DouweM · 2025-07-08T14:26:17Z

pydantic_ai_slim/pydantic_ai/models/instrumented.py

+        # Store instrumentation settings separately from model settings
+        self.instrumentation_settings = options or InstrumentationSettings()
+        # Initialize base Model with no settings to avoid storing InstrumentationSettings there
+        Model.__init__(self, settings=None)


If we make _settings a field on Model with default value None (like _profile currently is), we can skip this init.

DouweM · 2025-07-08T14:26:40Z

pydantic_ai_slim/pydantic_ai/models/mcp_sampling.py

@@ -42,6 +42,10 @@ class MCPSamplingModel(Model):
    [`ModelSettings`][pydantic_ai.settings.ModelSettings], so this value is used as fallback.
    """

+    def __post_init__(self):
+        """Initialize the base Model class."""
+        super().__init__()


We can likely also remove this if _settings: ModelSettings | None = None becomes a Model field.

DouweM · 2025-07-08T14:28:48Z

tests/models/test_model_settings.py

+    # Run with additional settings
+    run_settings = ModelSettings(temperature=0.8, seed=42)
+
+    # This should work without errors and properly merge the settings


Can we verify it ends up with the correct settings?

DouweM · 2025-07-08T14:29:30Z

tests/models/test_model_settings.py

+def test_gemini_model_settings():
+    """Test that GeminiModel can be initialized with settings."""
+    if not gemini_available or GeminiModel is None:  # pragma: no cover
+        return  # Skip if dependencies not available


Let's use pytest.skip here, there are some examples in the codebase

DouweM · 2025-07-08T14:30:28Z

tests/models/test_model_settings.py

+        return ModelResponse(parts=[TextPart('test')])
+
+    # Replace the instrumented model's wrapped model with a function model for testing
+    instrumented_model.wrapped = FunctionModel(test_response, settings=base_model_settings)


Let's create a new InstrumentedModel instead of reusing the existing one, as WrapperModel caches some properties so changing wrapped in place is not handled correctly across the board.

DouweM · 2025-07-08T14:32:18Z

docs/agents.md

+2. **Agent-level defaults** - Set during [`Agent`][pydantic_ai.agent.Agent] initialization via the `model_settings` argument. These override model defaults.
+3. **Run-time overrides** - Passed to `run{_sync,_stream}` functions via the `model_settings` argument. These have the highest priority and override both agent and model defaults.
+
+**Settings Precedence**: Run-time > Agent > Model


I think we can drop this line as the list already explicitly states the priority.

Actually, it may be useful to write here that the settings are merged, not replaced.

DouweM · 2025-07-08T14:34:01Z

docs/models/index.md

@@ -124,6 +124,39 @@ The `ModelResponse` message above indicates in the `model_name` field that the o
 !!! note
    Each model's options should be configured individually. For example, `base_url`, `api_key`, and custom clients should be set on each model itself, not on the `FallbackModel`.

+### Per-Model Settings
+
+You can configure different `ModelSettings` for each model in a fallback chain by passing the `settings` parameter when creating each model. This is particularly useful when different providers have different optimal configurations:


Suggested change

You can configure different `ModelSettings` for each model in a fallback chain by passing the `settings` parameter when creating each model. This is particularly useful when different providers have different optimal configurations:

You can configure different [`ModelSettings`][pydantic_ai.settings.ModelSettings] for each model in a fallback chain by passing the `settings` parameter when creating each model. This is particularly useful when different providers have different optimal configurations:

add settings field

cb9aa0f

fix tests

4a6ad50

DouweM self-assigned this Jul 7, 2025

DouweM requested changes Jul 7, 2025

View reviewed changes

pydantic_ai_slim/pydantic_ai/models/__init__.py Outdated Show resolved Hide resolved

DouweM added the awaiting author revision label Jul 7, 2025

jan-siml and others added 9 commits July 8, 2025 10:33

change to instrumentation_settings

df93aa4

Merge remote-tracking branch 'upstream/main' into Add-settings-to-Model

eff2619

fix tests

f8702c6

hide importerror from coverage

69aa087

format

c316d33

increase coverage

09b47dd

increase coverage

6d4dcc8

fix lint

4b1e44a

fix pragma

42622cf

DouweM requested changes Jul 8, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add `settings` to Model base class #2136

Add `settings` to Model base class #2136

Uh oh!

svilupp commented Jul 5, 2025 •

edited

Loading

Uh oh!

hyperlint-ai bot commented Jul 5, 2025

Uh oh!

DouweM left a comment

Uh oh!

Uh oh!

DouweM Jul 8, 2025

Uh oh!

DouweM Jul 8, 2025

Uh oh!

DouweM Jul 8, 2025

Uh oh!

DouweM Jul 8, 2025

Uh oh!

DouweM Jul 8, 2025

Uh oh!

DouweM Jul 8, 2025

Uh oh!

DouweM Jul 8, 2025

Uh oh!

DouweM Jul 8, 2025

Uh oh!

DouweM Jul 8, 2025

Uh oh!

DouweM Jul 8, 2025

Uh oh!

DouweM Jul 8, 2025

Uh oh!

DouweM Jul 8, 2025

Uh oh!

Uh oh!

	You can configure different `ModelSettings` for each model in a fallback chain by passing the `settings` parameter when creating each model. This is particularly useful when different providers have different optimal configurations:
	You can configure different [`ModelSettings`][pydantic_ai.settings.ModelSettings] for each model in a fallback chain by passing the `settings` parameter when creating each model. This is particularly useful when different providers have different optimal configurations:

Add settings to Model base class #2136

Are you sure you want to change the base?

Add settings to Model base class #2136

Uh oh!

Conversation

svilupp commented Jul 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Next Steps

Uh oh!

hyperlint-ai bot commented Jul 5, 2025

PR Change Summary

Uh oh!

DouweM left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Add `settings` to Model base class #2136

Add `settings` to Model base class #2136

svilupp commented Jul 5, 2025 •

edited

Loading